{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "placeholder是一个占位符，它通常代表着从外界输入的值。\n",
    "其中None代表着尚不确定的维度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = tf.placeholder(dtype=tf.float32, shape=[None, 2])\n",
    "y = tf.placeholder(dtype=tf.float32, shape=[None, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Variable是声明一个变量，变量的含义是在后面的优化过程中需要更新的权重、偏置等参数。\n",
    "目前它的维度需要是确定的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = tf.Variable(tf.zeros([2, 1]))\n",
    "b = tf.Variable(tf.zeros([1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们进行一系列的计算，得到logits，output和交叉熵。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "logits = tf.matmul(x, w) + b\n",
    "\n",
    "output = tf.nn.sigmoid(logits)\n",
    "\n",
    "cross_entropy = tf.losses.sigmoid_cross_entropy(multi_class_labels=y, logits=logits)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "这里我们创建了一个GradientDescentOptimizer类，并调用了minimize方法作为一个训练的step。\n",
    "minimize中包含两个操作compute_gradients和apply_gradients。\n",
    "注意我们并不在这里循环。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_step = tf.train.GradientDescentOptimizer(0.3).minimize(cross_entropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "train_step是一个op，并不会产生一个tensor（数值）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "name: \"GradientDescent\"\n",
      "op: \"NoOp\"\n",
      "input: \"^GradientDescent/update_Variable/ApplyGradientDescent\"\n",
      "input: \"^GradientDescent/update_Variable_1/ApplyGradientDescent\"\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(train_step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里我们定义输入的x和期望的y（ground truth）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_value = np.array(\n",
    "    [[1,1],\n",
    "     [1,0],\n",
    "     [0,1],\n",
    "     [0,0]])\n",
    "y_value = np.array(\n",
    "    [[1],\n",
    "     [1],\n",
    "     [1],\n",
    "     [0]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们虽然在w和b的声明中写了是zeros初始化，但是需要再次强调：\n",
    "在声明图的时候，我们其实并没有做任何的内存分配和实际的初始化操作。\n",
    "其实我们的init_op也仍然并没有做这个操作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "init_op = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "创建一个session"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这里，我们才真正初始化了相关的Variables。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess.run(init_op)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意我们在这里并没有执行train_step（尽管它也包含在graph中），所以不论执行多少次这个操作，w,b的值都不会改变。\n",
    "当然output和logits也就不会改变。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_entropy_value, logits_value, output_value = sess.run(\n",
    "    [cross_entropy, logits, output], \n",
    "    feed_dict={x:x_value, \n",
    "               y:y_value} )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6931472\n",
      "[[0.]\n",
      " [0.]\n",
      " [0.]\n",
      " [0.]]\n",
      "[[0.5]\n",
      " [0.5]\n",
      " [0.5]\n",
      " [0.5]]\n"
     ]
    }
   ],
   "source": [
    "print(cross_entropy_value)\n",
    "print(logits_value)\n",
    "print(output_value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "for current_step in range(100):\n",
    "    cross_entropy_value, output_value, _ = sess.run(\n",
    "        [cross_entropy, output, train_step], \n",
    "        feed_dict={x:x_value, \n",
    "                   y:y_value} )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_entropy_value, logits_value, output_value, w_value, b_value = sess.run(\n",
    "    [cross_entropy, logits, output, w, b], \n",
    "    feed_dict={x:x_value, \n",
    "               y:y_value} )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.21702278\n",
      "[[ 3.9128602 ]\n",
      " [ 1.7881696 ]\n",
      " [ 1.7881696 ]\n",
      " [-0.33652085]]\n",
      "[[0.9804082 ]\n",
      " [0.85670274]\n",
      " [0.85670274]\n",
      " [0.41665486]]\n",
      "[[2.1246905]\n",
      " [2.1246905]]\n",
      "[-0.33652085]\n"
     ]
    }
   ],
   "source": [
    "print(cross_entropy_value)\n",
    "print(logits_value)\n",
    "print(output_value)\n",
    "print(w_value)\n",
    "print(b_value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
